DOCUMENT RESUME 



ED 219 449 



TM 820 477 



AUTHOR 
TITLE 

INSTITUTION 

SPONS AGENCY 

REPORT NO 
PUB DATE 
CONTRACT 
NOTE 

EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Fuchs, Douglas; And Others 

A Microanalysis o£ Participant Behavior in Familiar 
and Unfamiliar Test Conditions. 

Minnesota Univ., Minneapolis. Inst, for Research on 
Learning Disabilities. 

Office of Special Education and Rehabilitative 

Services (ED) , Washington, DC. 

IRLD-RR-70 

Mar 82 

300-80-0622 

30p.; For related document, see ED 203 587 
MF01/PC02 Plus Postage. 

^Behavior Patterns; *Disabili ties; Examiners; 
♦Experimenter Characteristics; *Tnteraction Process 
Analysis; Preschool Education; ^Speech Communication; 
Test Coaching 

♦Familiarity; *Microanalysis 



ABSTRACT 

In a microanalysis of the behaviors of examiners and 
handicapped children during videotaped testing sessions, handicapped 
students performed better with familiar examiners than with 
unfamiliar examiners. The children spoke significantly more often and 
longer when tested by familiar examiners who exercised more frequent 
and longer intervals of silence, and appeared^ to use eye contact with 
examinees as a cue in deciding when to speak. Unfamiliar examiners 
rarely utilized this cue. Familiar examiners also employed largely 
directive language in contrast to unfamiliar examiners* speech which 
was more frequently participatory in nature and longer in duration. 
The clues to understanding why differences between familiar and 
unfamiliar examiners* behavior affects test performance, and the 
implications for testing handicapped children are discussed. Silence 
is shown to be an important examiner behavior over which 
test-developers and clinician*trainers should exercise greater 
control . ( Author/CM) 
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Abstract 



A microanalysis was conducted of the behaviors of examiners and 
handicapped children during videotaped testing sessions in which handi- 
capped students performed better with familiar than with unfamiliar 
examiners. The children spoke significantly more often and longer 
when tested by familiar examiners. Familiar examiners (a) exercised 
more frequent and longer intervals of silence than unfamiliar examiners, 
(b) often appeared to use eye contact with examinees as a cue in decid- 
ing when to speak whereas unfamiliar examiners rarely utilized this 
cue, (c) employed largely directive language in contrast to unfamiliar 
examiners' speech that more frequently was participatory in nature, and 
(d) spoke for a shorter duration than unfamiliar examiners. These 
differences between familiar and unfamiliar examiners* behavior provide 
clues for understanding why differential test performance occurred. Im- 
plications for testing handicapped children are discussed. 
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A Microanalysis of Participant Behavior 
in Familiar and Unfamiliar Test Conditions 

Over the past 40 years, research has shown that extra-test varia- 
bles, such as situational and procedural factors, often influence test 
performance (Masling, 1960; Sattler, 1981; Sattler & Theye, 1967). 
Similarly, investigation of examiner characteristics has demonstrated 
that a tester's personal familiarity with select populations, such as 
young and handicapped children, enhances the examinee's test performance 
(Fuchs, Featherstone, Garwick, & Fuchs, 1981; Fuchs, Garwick, Featherstone, 
& Fuchs, 1980; Olswang & Carpenter, 1978; Stoneman & Gibson, 1978). 
However, previous research appears to regard the influence of examiner 
familiarity, as well as the effects of other examiner characteristics 
such as race, sex, personality, and professional experience, as self- 
explanatory (Masling, 1960; Sattler, 1981); it fails to explore whether, 
and if so how, familiar and unfamiliar examiners behave differently from 
one another, why they act as they do, and to what extent, if any, their 
actions and motives are shaped by examinee's characteristics and test 
behaviors . 

Contributing to the salience of these questions is that many young 
children currently are tested by unfamiliar adults in school and clinical 
settings. If the purpose of such testing is to measure and describe 
children's optimal performance, we need to understand how and why this 
differential functioning exists before we may hope to vitiate the negative 
effect that examiner unfamil iarity exerts on certain children's perform- 
ance. 
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The present exploratory investigation was a microanalysis of video- 
taped testing sessions of handicapped children who performed stronger 
with familiar than unfamiliar examiners within a repeated measures 
experiment. The purpose of this analysis was to delineate examinee and 
examiner behaviors that differed between the two experimental conditions 
in hopes of understanding why differential test performance occurred. 
Toward this end, the frequency, duration, and quality of examiner and 
examinee speech and silence were contrasted across familiar and unfamiliar 
conditions. These facets of the testing situation were selected with 
the recognition that the examiner-examinee relationship is bi-directional 
and dynamic. This is in contrast to the view, offered implicitly by 
much of the prior research on examiner effects, that the exeminer-examinee 
relationship is uni-directional and static (cf. Mehan, 1973, Roth, 1974). 

Method 

Subjects 

Subjects were 15 examinees and six examiners. The examinees were 
moderately to profoundly speech and/or language handicapped students 
enrolled in a special education program within a large urban school 
system in the Midwest. They were a subsample from a previous study 
(Fuchs, Dailey, & Fuchs, 1982; Fuchs et al., 1980), in which subjects 
displayed greater descriptive accuracy, verbal fluency, and linguistic 
complexity in discussing a picture stimulus when tested by a familiar 
examiner than when tested by an unfamiliar examiner. Examinees were 
representative of the larger sample in terms of (a) age (R = 4.7), (b) 
sex (60% male), (c) race {20% black and 12% American Indian), (d) 
length of time that they had been in their present classrooms (5( = 8.1 
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months), and (e) the average direction and magnitude of their differen- 
tail performance in the two examiner conditions. 

Familiar examiners (n^ = 2) were the classroom teachers of the ex- 
aminees, and, as such, they shared a relatively long-term acquaintanceship 
with the examinees. Unfamiliar examiners (n = 4) were strangers to the 
examinees. All examiners were female, certified in early childhood edu- 
cation, and had at least several years o^ experience working with pre- 
school children in educational settings. 
Procedure 

Videotaping and editing . The students* test performance with 
familiar and unfamiliar examiners was videotaped by two AVC 3200 Sony 
video cameras on one-half inch videotape. The cameras, connected to 
a Sony 3600 recorder, were placed behind the examiner and examinee. 
With the aid of a special effects generator (SEG-1), a split screen 
was created displaying a frontal view of the upper torso and head of 
both adults and children. With the video cameras, recorder, and special 
effects generator placed behind blinds, the only visible recording 
equipment were two table microphones. Examiners were aware of the 
recording effort; students were not. After the completion of all 
recording, the videotaped testing sessions were edited electronically 
with a Panasonic 316C editing tape recorder so that each pupiTs recorded 
performance with familiar and unfamiliar examiners was contiguous. 

Experimental task . The videotaped ex .rimental task was similar 
to certain test items on the Stanford-Bin^ It required examinees to 
describe verbally a high-interest, representatio.ial drawing that was 
selected from Tester's (1966) Teaching Pictures series. An explicit 



format governed the administration of this task. Examiners were instructed 
to speak only three times: to introduce the task, to prompt the examinee 
for additional information, and to communicate termination of the task. 
While the content and frequency of examiners' talk was controlled, their 
timing, or when during the test session they spoke, was uncontrolled 
purpose'^y. (For a more detailed description of the experimental task 
and design of the prior study, see Fuchs et al., 1980.) 

Videotape analysis . Examiner and examinee behavior was divided 
into three general categories: examiner talk (E), examinee talk (C), 
and examinee's visual behavior during silent intervals. This last 
category was subdivided further into: looking at the stimulus picture (P)^ 
looking at the tester (T), and looking somewhere other than P or T (0). 

The videotapes were analyzed by two undergraduate student observers 
who did not know the purposes of the study, any of the examiners or 
examinees, or whether testing sessions were familiar or unfamiliar. 
The observers were required to transcribe the videotapes into the five 
behavioral categories, displaying both the temporal sequence of these 
behaviors and their duration. The observers were trained to code the 
tapes in three one-hour sessions by a graduate student who, like the 
observers, knew nothing about the study. Inter-rater reliability was 
calculated on 11 of 30 test sessions (15 examinees x 2 conditions), or 
37' of the data. Inter-rater agreement ranged from .75 to 1.00, with 
an average reliability of .91. 
Data Analj^sis 

Examiner and student behaviors were analyzed in terms of both single 
behaviors (P, 0, T, E) and behavior couplings (e.g., PC, PT, PE). 
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Chains of three or more behaviors were divided into contiguous couplings 
by pairing adjacent behaviors in overlapping fashion. A chain of PTPT, 
for example, was separated into PT, TP, PT. This resulted in 14 behav- 
ioral categories, including both single behaviors and couplings. 
These categories, in turn, provided the frame for a matrix that facili- 
tated the exploration of antecedents to examiner and examinee talk. 
Since the primary purpose of the study was to identify events that were 
antecedent to test participants' speech, analyses excluded behavior that 
followed the last utterance of the child in the test session. A chi 
square analysis and correlated t^ tests were used to compare examiner 
and examinee behavior across familiar and unfamiliar conditions. 

Resul ts 

Examinee Talk 

In the familiar condition, the students spoke more often, t{]^) = 
2.20, £ < .05, and for greater amounts of time, t(14) = 2.37, £ < .05 
(see Table 1). However, when the amount of examinee talk was divided 
by the frequency of their utterances, no difference was obtained 
between familiar and unfamiliar conditions (see Table 1). 

Insert Table 1 about here 

Examiner Tal k 

No statistically significant difference was found between the 
frequency of familiar and unfamiliar examiner's talk. Yet, as shown 
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in Table 2, unfamiliar testers spoke for greater amounts of time, t(14) = 
2.32, £ < .05. When the amount of examiners' talk was divided by the 
frequency of their speech, the difference between familiar and unfamiliar 
testers was not significant. 

Insert Table 2 about here 

Unfamiliar evaminers deviated from test instructions by uttering 
11 unsanctioned messages to seven examinees; familiar testers displayed 
five instances of this rule violation during the testing of three examinees. 
W. lie the unapproved talk of the familiar examiners seemed to be largely 
directive (e.g., "Tell me more,** "Point to what you mean," Sit down"), 
much of the unfamiliar examiners' unsanctioned speech appeared to have 
a participatory quality (e.g., "Oh yeah?" "Really?" "Hmm!" "Do you want 
to get up like those girls [in picture]?" "Right. It's [oicturel a 
bey and a doctor. " ) . 
Si lence 

Total silence . Table 3 displays the raw scores for the mean fre- 
quency and duration of silent intervals in familiar and unfamiliar 
conditions. In the familiar situation, silent intervals were more num- 
erous, t(14) = 4.99, £ < .001, and the total duration of silence was 
greater, t(14) - 3.43, £ <- .01 . 

Insert Table 3 about here 
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Associated with the higher frequency of silent intervals in the 
familiar condition was a greater variance (S : 25.10 vs. 5.21), t(13) 
= 4.65, £ < .01.^ With respect to the total duration of silence, there 
was no significant difference between t.ie variances associated with 
the two conditions (familiar: S = 56.68; uifamiliar: S = 31.72). 
Also, between familiar and unfamiliar situations, there was no signi- 
ficant disparity in the mean duration (duration/frequency) of silent 
intervals (see Table 3) . 

Silence before e xaminee talk . The frequency with which examinee 
and examiner behavior preceded examinee talk (C) was divided by the 
number of times an examinee spoke, thereby controlling for the greater 
frequency of C in the familiar condition. Ratios for verbal and non- 
verbal behavior in the unfamiliar situation then were subtracted from 
corresponding behavior in the familiar condition. Table 4 displays the 
findings from these contrasts for individual cases and for the averages 
across examinees. 

Insert Table 4 about here 

As shown in this table, two contrasts, both involving T-related 
behavior (looking at tester), were significant. First, the ratio of 
TP (looking at tester then at picture) preceding C was greater in the 
familiar than unfamiliar situation, t(13) - 2.22, £ - .05 (mean difference 
= + .16). Second, the ratio of TE (looking at tester then examiner talk) 
that was antecedent to C was greater in the familiar condition, t(13) 
- 2.97, £ .02 (mean difference = f.l9). Table 4 also indicates that 
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a combination of Pf and TP preceded C more in the familiar situation 
(mean difference = +.30), whereas, in the unfamiliar condition, PE and 
EP preceded C more (mean difference = -.22). Additionally, there was a 
significant relation between exarr.iner condition arid T- and E-related 
behaviors that were antecedent to C, x2{l) = 8.98, £ - .005. 

Si lence before examiner talk . A total of 73'^ of familiar examiners* 
sanctioned speech was preceded immediately by the student looking at the 
examiner; only 27% of such speech was preceded by the student looking at 
the stimulus picture. In contrast, for unfamiliar examiners, ^0% of 
sanctioned speech was preceded by the student looking at the examiner and 
43% was preceded by the student Tookinq at the picture. 

Discussion 

A previous study (Fuchs et al., 1981; Fuchs et al., 1980) demonstrated 
that handicapped preschool children employed (a) more words, (b) more 
complex syntactic and semantic structures, and (c) greater descriptive 
accuracy in discussing a picture when tested by familiar examiners than 
when tested by unfamiliar examiners. Employing a representative sub- 
sample of the subjects invoU in that study, this investigation re- 
analyzed the children's verbal fluency by examining the duration as well 
as the frequency of their speech, and further by locating their talk in 
relation to their visual behavior during silent intervals and to examiner 
speech. This microanalysis was undertaken to identify the antecedents 
to examinee speech in familiar and unfamiliar conditions in order to 
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u Jerstand better why differential test performance occurred. 

Examiner talk, a logical antecedent to children's speech in a test 
situation, appeared to be unrelated to the examinees' greater fluency 
in the familiar condition. Familiar examiners did not speak more fre- 
quently than unfamiliar examiners; in fact, the total duration of 
familiar examiners' talk was less than that of unfamiliar examiners' 
speech. However, both the frequency of silent intervals and the total 
duration of silence preceding examinee talk were associated positively 
with examinees' more frequent and extended verbal production in the 
fami 1 iar situation . 

The present study suggested that examiner behavior contributed to 
the greater frequency of silent intervals observed in che familiar 
situation. Analysis of apnarent cues used by examiners in the determina- 
tion of when to speak revealed that familiar examiners were significantly 
more likely rhan unfamiliar examiners to speak only after examinees had 
looked up from the stimulus picture to look at them; unfamiliar examiners 
displayed a tendency to speak more often than familiar examiners while 
the children still were viewing the stimulus. These disparate patterns 
of examiner behavior were reflected by the fact that TP and PT couplings 
preceded C 2.3 times more frequently in the familiar condition, whereas 
EP and PE couplings were antecedent to C 1 .7 times more often in the 
unfamiliar condition. Similarly, the pattern TPC--examinee looking at 
the tester, at the stimulus picture, and then speaking--was significantly 
more frequent in the familiar condition. 
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Two points need to be made both with respect to the findings and 
interpretations developed from them. First, due to the relatively large 
number of correlated t^ tests used and the inter-relatedness of much of 
the data, findings may reflect Type I error. Second, this was a cor- 
relative study rather than an experimental investigation; demonstrated 
relationships between examiner anc^ examinee behavior are not necessarily 
causative. 

Nevertheless, findings were consonant with the supposition that 
the greater amount of and more frequent silence preceding exam' ler calk 
in the familiar condition contributed to examinees* greater verbal fluency 
in the familiar situation. Findings also suggested that examiner behavior 
was at least partly responsible for this silence. The data from this 
study are insufficient to explain the obtained differences in the 
behavior of familiar and unfamiliar examiners. It is reasonable to 
speculate, however, that the unfamiliar examiners' relatively infrequent 
use of silence (i.e., their apparent proneness to terminate the testing 
in an untimely manner) was related to their ignorance of examinees' skill 
level and knowledge base. Confronted by examinees' silence, unfamiliar 
examiners had to decide whether to ignore this silence and encourage 
continued effort or to withdraw the test demand. Presumably, this 
decision normally is based on an understanding of the examinees' capa- 
bilities. Because unfamiliar examiners in this study had minimal in- 
formation about examinees' ability levels, were presumably perceptive 
about and empathetic towards children's feelings, and were no doubt 
aware of their need for examinees' cooperati veness , they may have believed 

I ^ 
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there was no alternative but to behave conservatively and employ examinees' 
silence as a primary cue in determining when to conclude testing. 

Consonant with the speculation that unfamiliar testers were compara- 
tively uneasy during assessnent were findings that they spoke to exam- 
ees for a significantly greater duration of time than familiar testers 
and that their relatively more frequent unsanctioned speech appeared 
to be participatory in nature. 

In contrast to unfamiliar testers, familiar examiners, by defini- 
tion, had a more accurate notion of examinees' capabilities. When 
testing children who were reticent, yet known to be capable, familiar 
testers appeared comparatively unresponsive to examinee silence, and 
siippositively communicated an expectation that the children demonstrate 
their potential. In agreement with this interpretation, familiar exam- 
iners were more variable in their deployment of silence than were un- 
familiar testers. Also, because familiar testers shared a relatively 
long acquaintanceship with the examinees, the familiar examiners may 
have been less concerned than urifamiliar examiners about examinees' 
discontinued cooperation during assessment. 

If testers' differential use of silence at least partially explains 
why select groups of children tend to perform more strongly with famil- 
iar than with unfamiliar examiners, then silence is an important examiner 
behavior over which test developers arJ clinician-trainers should exercise 
greater control. Future research might use experimental designs to 
explore the importance of silence to Cest performance. On the basis 
of this investigation, one might hypothesize that if familiar and un- 
familiar examiners were required to employ silence similarly, the 
negative effects of examiner unfamil iarity may be vitiated. 

ERLC 
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At a more general level, familiar and unfamiliar examiners' dif- 
ferential use of silence (as well as other differences in participants* 
behavior between familiar and unfamiliar conditions) corroborates the 

view that examiners and examinees interact in a bi-directional, dynamic, 
and creative process; in a sense, a child's test performance is a social 
accomplishment to which both examinee and examiner may lay claim (see 
Mehan, 1978). Simultaneously, findings from this study question the 
still popular Galtonian view that a test is no more or less than a 
sample of the subject's responses to a standardized non-personal stimulus 
The present study and past investigations suggest that such a view may 
interfere with both securing optimal test performance and developing 
meaningful educational programs that are based upon formal assessment 
resul ts . 
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Footnotes 



Douglas Fuchs also is a Postdoctoral Fellow at the Institute for 
Research on Learning Disabilities. 

^The following formula (MacEachern, 1982) was used for testing the 
difference between two variances that were calculated on the same 
sample: 



{s/-s/)Jn:2 



t = 
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Table 1 

Amounts of Examinee Talk with Familiar and 
Unfamiliar Examiners^ 





Examiner 


Condition 


Analysis 


Familiar 


Unfamil iar 


Frequency* 


3.80(2.43) 


2.33(1 .29) 


Duration* 


9.53(8.68) 


5.83(4.90) 


Duration/ Frequency 


2.60(2.40) 


2.35(1 .68) 



^Entries are means and standard deviation? (in parentheses) of the 
number of times the 15 examinees spoke (Frequency) and the number of 
seconds they spoke (Duration). 

★ 

Difference between Familiar and Unfamiliar conditions was 
statistically significant (p <^ .05). 
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Table 2 

Amounts of Examiner Talk in Familiar and 
Unfamiliar Conditions^ 



Examiner Condition 



Analysis 


Famil iar 


Unfamil iar 


Frequency 


1.33(.82) 


1 .80(1 .15) 


Duration* 


1.28(.92) 


2.71(1.91) 


Duration/Frequency 


.90(.25) 


1 .56(1.01) 



^Entries are means and standard deviations (in parentheses) of the 
number of times examiners spoke (Frequency) and the number of seconds 
they spoke (Duration). 

'^Familiar examiners N = 2; Unfamiliar examiners N = 4. 

Difference between Familiar and Unfamiliar conditions was statistically 
significant (£ < .05) . 
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Table 3 

Silence in Familiar and Unfamiliar Con'^itions 





Examiner 


Condition 


Analysis 


Famil iar 


Unfamil iar 


Frequency*** 


9.60(4.69) 


5.07(2.28) 


Duration** 


13.78(7.53) 


6.01(5.63) 


Duration/Frequency 


2J7( .57) 


2.24(1 .29) 



^Entries are the means and standard deviations (in parentheses) of the 
number of instances of silence (Frequency) and their length in 
seconds ( Duration) . 
**Difference between Familiar and Unfamiliar conditions was statistically 

significant (£ < .01 ) . 
***Difference between Familiar and Unfamiliar conditions was statistically 
significant (£ <^ .001). 



Table 4 

Ratios of Behavior Preceding Examinee Talk for Familiar Minus Unfamiliar Conditions' 



s c 
Examinees Behavior Categories 





P 


0 




T 


E 


PO 


OP 


PT 


TP 


PE 


EP 


OE 


EG 


TE 


ET 




1 


+ .25 










-.50 


-.50 


+ .75 


+ .50 


-.50 


+ . 


25 






+ .25 


-.50 




2 


+ .75 


















-1 .00 




75 






+ .25 






3 


-.25 




























+ .25 






4 


+ .25 
















+ .25 


+ .25 


+ 


25 






+ .25 


-.25 




5 


-.24 














+ .38 


+ .29 












-.19 


+ .25 




6 


+ .25 








-.50 


























7 


-.50 




















+ 


50 












8 


+ .25 






-.25 












-.25 




25 


-.50 


-.25 


+ .50 






9 


-.21 














-.36 


+ .57 


+ .14 




.36 












10 


+ .67 


















-.50 


+ 


.33 






+ .33 






11 


-.50 














+1.00 


+ .50 




+ 


.50 






+ .50 






12 


.00 






















.00 






.00 






13 


+ .20 




10 




+.10 






-.33 


-.33 






.57 


+ .10 


+ .10 


-.23 






14 


-.25 














+ .50 


+ .50 






.25 






+ .25 


+ .25 






+ .05 




03 


..02 


-.03 


-.04 


-.04 


+ .14 


+ .16* 


-.19 




.03 


-.03 


-.01 


+.19** 


-.02 




SO 


.39 




10 


.07 


.14 


.13 


.13 


.38 


.27 


.52 




.43 


.14 


.07 


.24 


.18 





^Frequency of each behavior was divided by the frequency of examinee talk. Spaces in table reflect absence 
of behavior preceding examinee talk and are treated as zeroes in calculating means and standard deviations. 

^One examinee was excluded because of no talk in the unfamiliar condition. 
^P = Examinee looks at picture; 0 = Examinee looks at something other than picture or tester; 
T = Examinee looks at tester; E = Examiner talk. 



♦Difference between Faniliar and Unfamiliar conditions was statistically significant (£ < ■05). 
♦♦Difference between Familiar and Unfaniliar conditions was statistically significant (p_ < .01). 
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